[None][test] Update QA test list of RTX 6K. by pamelap-nvidia · Pull Request #14421 · NVIDIA/TensorRT-LLM

pamelap-nvidia · 2026-05-21T15:35:28Z

Summary by CodeRabbit

Tests
- Streamlined test suite configurations by consolidating Llama 4 model tests to specific GPU configurations and removing redundant test entries for encoder/multimodal and disaggregated-serving components.

Description

Reduce the test list of RTX 6K

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

coderabbitai · 2026-05-21T15:37:16Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 92c80201-d1af-43dc-afb9-bf89428af317

📥 Commits

Reviewing files that changed from the base of the PR and between 075933a and b76aabb.

📒 Files selected for processing (1)

tests/integration/test_lists/qa/llm_function_rtx6k.txt

💤 Files with no reviewable changes (1)

tests/integration/test_lists/qa/llm_function_rtx6k.txt

📝 Walkthrough

Walkthrough

The PR reduces the RTX 6K QA test list by removing 52 test entries across four changes: eliminating two accuracy tests, consolidating Llama4 variants to specific tensor/expert parallel configurations, removing encoder and multimodal tests, and dropping disaggregated-serving tests for TinyLlama.

Changes

QA Test List Curation

Layer / File(s)	Summary
RTX 6K test entry removals and consolidation `tests/integration/test_lists/qa/llm_function_rtx6k.txt`	The test list removes DeepSeek/Nemotron/Minitron accuracy entries, replaces a wide range of Llama4 Maverick/Scout configurations with only tp8ep8 and tp4ep4 variants using cuda_graph, eliminates encoder encoding and multimodal accuracy tests, and removes disaggregated-serving tests for TinyLlama.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

NVIDIA/TensorRT-LLM#10210: Both PRs modify tests/integration/test_lists/qa/llm_function_rtx6k.txt by adding/removing and regrouping RTX 6K LLM QA test case entries.
NVIDIA/TensorRT-LLM#14335: Both PRs remove the deprecated disaggregated-serving test_disaggregated_trtllm_sampler[TinyLlama-1.1B-Chat-v1.0] test and its test list references.

Suggested reviewers

crazydemo
StanleySun639
LarryXFly

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	❓ Inconclusive	The PR description only contains a template with 'Reduce the test list of RTX 6K' as description, but lacks key sections like detailed explanation of why changes were made, test coverage details, and incomplete checklist items.	Expand the description to explain why the test list is being reduced and which specific tests were removed, provide details on test coverage validation, and ensure the PR checklist is fully reviewed.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: updating the QA test list for RTX 6K, with proper ticket format [None], type [test], and specific focus.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

⚔️ Resolve merge conflicts

Resolve merge conflict in branch 6kd_testlist

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

pamelap-nvidia · 2026-05-22T00:12:58Z

/bot run --skip-test

tensorrt-cicd · 2026-05-22T00:19:08Z

PR_Github #49795 [ run ] triggered by Bot. Commit: b0554bd Link to invocation

Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>

tensorrt-cicd · 2026-05-22T02:14:29Z

PR_Github #49795 [ run ] completed with state SUCCESS. Commit: b0554bd
/LLM/main/L0_MergeRequest_PR pipeline #39385 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>

jieli-matrix · 2026-05-22T03:13:09Z

/bot run --stage-list ""

tensorrt-cicd · 2026-05-22T03:20:14Z

PR_Github #49834 [ run ] triggered by Bot. Commit: cee8beb Link to invocation

tensorrt-cicd · 2026-05-22T04:00:03Z

PR_Github #49834 [ run ] completed with state SUCCESS. Commit: cee8beb
/LLM/main/L0_MergeRequest_PR pipeline #39419 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation

crazydemo · 2026-07-15T04:12:45Z

I think this PR is out-of-date, @pamelap-nvidia Could you please help confirm if we can close this PR for now?

xinhe-nv · 2026-07-20T02:23:20Z

 accuracy/test_llm_api_pytorch.py::TestLlama3_3_70BInstruct::test_fp8_eagle3_tp8[eagle3_one_model=False-torch_compile=False]
 accuracy/test_llm_api_pytorch.py::TestLlama3_3_70BInstruct::test_fp8_eagle3_tp8[eagle3_one_model=False-torch_compile=True]
 accuracy/test_llm_api_pytorch.py::TestPhi4MiniInstruct::test_auto_dtype
+accuracy/test_llm_api_pytorch.py::TestLlama4MaverickInstruct::test_auto_dtype[tp8ep8-cuda_graph=True]


llama4 is obsoleted, do we still need to test it?

crazydemo · 2026-08-03T05:58:15Z

close this MR, as it is out-of date

pamelap-nvidia requested a review from a team as a code owner May 21, 2026 15:35

github-actions Bot assigned pamelap-nvidia May 21, 2026

Update llm_function_rtx6k.txt

55aa6e7

Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>

pamelap-nvidia force-pushed the 6kd_testlist branch from b0554bd to 55aa6e7 Compare May 22, 2026 01:00

pamelap-nvidia requested a review from jieli-matrix May 22, 2026 01:02

fix pre-commit

1660e7f

Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>

jieli-matrix approved these changes May 22, 2026

View reviewed changes

Comment thread tests/integration/test_lists/waives.txt

jieli-matrix requested review from StanleySun639, crazydemo and xinhe-nv May 22, 2026 02:31

revert changes in waives.txt

cee8beb

Signed-off-by: Pamela <179191831+pamelap-nvidia@users.noreply.github.com>

jieli-matrix force-pushed the 6kd_testlist branch from cee8beb to 1cbc11a Compare May 22, 2026 03:12

pamelap-nvidia force-pushed the 6kd_testlist branch from 1cbc11a to cee8beb Compare May 22, 2026 03:20

StanleySun639 approved these changes Jul 15, 2026

View reviewed changes

xinhe-nv reviewed Jul 20, 2026

View reviewed changes

crazydemo closed this Aug 3, 2026

Uh oh!

Conversation

pamelap-nvidia commented May 21, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

coderabbitai Bot commented May 21, 2026

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Suggested reviewers

❌ Failed checks (1 inconclusive)

Uh oh!

pamelap-nvidia commented May 22, 2026

Uh oh!

tensorrt-cicd commented May 22, 2026

Uh oh!

tensorrt-cicd commented May 22, 2026

Uh oh!

Uh oh!

jieli-matrix commented May 22, 2026

Uh oh!

tensorrt-cicd commented May 22, 2026

Uh oh!

tensorrt-cicd commented May 22, 2026

Uh oh!

crazydemo commented Jul 15, 2026

Uh oh!

xinhe-nv Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

crazydemo commented Aug 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pamelap-nvidia commented May 21, 2026 •

edited by coderabbitai Bot

Loading